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Requirement Name 

Original 
Amount of 
Test Hours 

Final 

Amount of 
Test Hours 

Attitude & 
Navigation 
Performance 

32 

112 

Cold Start 1 

2.5 

12 

Warm Start 1 

2.5 

12 

Maximum Velocity 
& Attitude Rate 2 

11.25 

14.25 

Master Antenna 
Switching 2 

7.5 

15 

Almanac Upload 
& Download 2 

7.5 

15 

Health Message 2 

7.5 

7.5 

Satellite Selection 
(Manual & Auto.) 2 

7.5 

7.5 


1 Each test run was 30 minutes in length and a total of 5 
runs. 

2 Each test run was 30 minutes in length and a total of 15 
runs. 

Table 1: A summary of the SIGI receiver requirements 
tested with initial and final number of test hours. 

is canted 41.5° away from the zenith direction[4], This 
greatly impacts the ability of the receiver to track enough 
GPS Satellite Vehicles (SV’s) to produce navigation and at- 
titude solutions. Since construction of the ISS will be com- 
pleted on-orbit, a self survey will not be possible. There- 
fore, this receiver utilizes a double difference attitude de- 
termination algorithm which eliminates the need to pre- 
calibrate the receiver using self-survey methods. 

The ISS program had 20 different GPS only require- 
ments that had to be verified by engineers at the GSFC 
GPS Test Facility. These requirements included not only 
performance specifications for navigation and attitude per- 
formance but also several functional requirements. These 
functional requirements are summarized in the first column 
of Table 1. 

MINIMUM NUMBER OF RUNS 

The first question every manager asks a test engineer 
is how long will it take to prove that a receiver is ready 
for delivery to the customer. The answer to this question 
varies greatly depending who is answering and how much 
experience the customer has had with flying GPS receivers. 
The first impression that our test team had of the number 
of runs for assessing navigation and attitude performance 
was somewhere between six and ten hours for each test 
run. Four or five test runs was believed to be sufficient, 
especially if each run was completely different from the 
other 4 runs (different almanacs and start times). It was 
also believed that other tests of only 45 minutes would be 


sufficient to test specific requirements such as: time to first 
position fix (TTFF), time to first attitude solution (TTFA), 
performance over an end of week rollover, performance at 
an altitude above the normal ISS altitude of 300 nautical 
miles. The complete list of requirements, the originally es- 
timated amount of time for testing that requirement and the 
final amount of time for testing that requirement is found 
in Table 1. 

The reader will discover that initially estimated number 
of test hours to adequately characterize the performance of 
this new receiver was considerably less than the final total 
of hours. This was a lesson that was learned by many hours 
of restarting simulations, tracking down small errors in the 
simulation configuration, or small errors in the operation of 
the receiver. Some errors were due to problems with the re- 
ceiver but the majority of problems, during the tests, were 
due to incorrect simulation and receiver configuration set- 
tings that would produce the undesireable results. Trying 
to determine where the particular problem was in the test 
setup was the major reason for some many test runs. With 
so many new parameters being tested at the same time, it 
was difficult to isolate the true source of the problem. In the 
future, a Force- 19 receiver running in parallel with other 
new receivers to provide a benchmark on the behavior of 
the simulator and the new receiver will be a benefit. 

Prior to the start of testing, a methodical attempt at de- 
termining the minimum number of test runs was performed 
using statistical sampling techniques. The first issue was 
the number of simultaneously changing parameters during 
test runs designed to just look at position, velocity, and at- 
titude performance. If a spacecraft with a GPS receiver is 
orbiting at an altitude of 250 nautical miles and a constella- 
tion of GPS SV’s are orbiting at above 9500 nautical miles 
in different orbit planes, the ability to isolate many of the 
variables so that one or only a few parameters are chang- 
ing at a time is very difficult. In honesty, the most practical 
solution to this aspect of solving for minimum number of 
runs was to run several very long runs (the more, the bet- 
ter). However, there were many other requirements which 
could be tested by looking for a sample set of mean values. 

The minimum numbers of tests needed to test require- 
ments containing conditions for an event within a mean 
amount of time or with a certain success rate was calcu- 
lated. In terms of determining the sample size for the ISS 
requirements, two types of tests remain: tests with a mean 
parameter, tests with a success rate parameter. 

SAMPLE SIZE OF A MEAN 

The first test type deals with a requirement that has a 
time varying component. For example the calculation for 
the number of samples to prove a requirement such as Time 
to First Fix (TTFF), three pieces of information are needed. 
First the confidence interval is chosen, which in this case 
is 95%. This confidence interval is then expressed in Z rt / 2 
coefficient for a Normal Distribution. This Z a / 2 coefficient 
can be found in the back of most Statistics books. Next an 
initial guess of the standard deviation of the time required 




Max. 
Error 
of Est. 

Z ( \/2 

in % 

Zctj 2 

Std. 

Dev. 

Sample 

Size 

n 

1 

99 % 

2.575 

10 

663 

1 

95 % 

1.96 

10 

384 

1 

90% 

1.645 

10 

271 

5 

99% 

2.575 

10 

27 

5 

95 % 

1.96 

10 

15 

5 

90% 

1.645 

10 

11 

10 

99 % 

2.575 

10 

7 

10 

95 % 

1.96 

10 

4 

10 

90% 

1.645 

10 

3 

30 

99% 

2.575 

10 

1 

30 

95% 

1.96 

10 

0 

30 

90% 

1.645 

10 

0 


Table 2: A listing of sample run sizes for a given confidence 
interval Z a / 2 

the receiver’s best performance possible given better than 
actual conditions. The sinusoidal error was an increase in 
position error over time because the millisecond value on 
the clock bias was not used to correct the timetag for the 
position solution. This slowly growing error was not no- 
ticeable until roughly six hours into the run. 

Another example of why long test runs are needed oc- 
curred when the customer (Johnson Space Center) performed 
a 24 test run. At 17 hours into the run, the receiver stopped 
producing attitude, navigation, dropped all tracked GPS 
SV’s and then reset itself. The details of why this prob- 
lem occurred and the solution to the problem is discussed 
in more detail in the next section. 

A RECEIVER PROBLEM SEVENTEEN 
HOURS INTO A RUN 

A good example of how testing the Force- 19 receiver 
for long periods of time can uncover a very specific and 
yet critical condition was during a 24 hour test run. At 
17:10 into this test run, the GPS receiver began to lose sig- 
nal lock on all GPS SV’s, lost attitude and navigation state 
knowledge and then performed a power cycle to reset its 
internal memory. This simulator test was first built by the 
customer, originally executed by the customer, and once 
the problem occurred, brought to our attention. The initial 
reaction was, of course, to blame something other the GPS 
SG or the GSFC portion of the internal firmware. The next 
step was to recreate the problem with the test run starting at 
just a few minutes prior to 17:10 into the run rather retest- 
ing the receiver for a full 17 hours. Fortunately this was 
done and the receiver did reproduce the exact same con- 
ditions for the short run as was witnessed during the 17 
hour test run. The most immediate navigation parameter 
which was obviously out-of-spec was that the PDOP value 
increased from an average of 4 to over 1700 in just 60 sec- 
onds. This was the first clue that the problem might lie 


Max. 
Error 
of Est. 

z a f 2 

in % 

Z (\/2 

Std. 

Dev. 

Sample 

Size 

n 

0.01 

99% 

2.575 

0.99 

656 

0.01 

95 % 

1.96 

0.99 

380 

0.01 

90% 

1.645 

0.99 

268 

0.02 

99% 

2.575 

0.99 

164 

0.02 

95% 

1.96 

0.99 

95 

0.02 

90% 

1.645 

0.99 

67 

0.03 

99% 

2.575 

0.99 

73 

0.03 

95 % 

1.96 

0.99 

42 

0.03 

90% 

1.645 

0.99 

30 

0.04 

99% 

2.575 

0.99 

41 

0.04 

95% 

1.96 

0.99 

24 

0.04 

90% 

1.645 

0.99 

17 

0.05 

99% 

2.575 

0.99 

26 

0.05 

95 % 

1.96 

0.99 

15 

0.05 

90% 

1.645 

0.99 

11 


Table 3: A listing of sample run sizes when testing for a 
success rate (pass or fail) and a confidence interval (Z^/ 2 ) 

in the environment being presented to the receiver. How- 
ever, no concrete evidence of that fact was available. There- 
fore the receiver internal software was recompiled to run in 
a “debug” mode where low-level status information about 
the firmware would be more available than is usually pro- 
vided to the users. This did provide some clues as to the 
possible reasons for the power cycle of the receiver. Some 
suggestions were made for software changes to both the 
navigation and attitude firmware to survive an unplanned 
power cycle during the mission. At the same time, a deeper 
look at the environment the receiver was presented by the 
GPS SG was made by test engineers. Upon closer inspec- 
tion of the settings of the receiver and the GPS SG, it was 
noticed that the GPS receiver was commanded to look for 
GPS SV’s down to an elevation mask of 0° while the GPS 
SG was commanded to simulate the constellation to an el- 
evation mask of 5°. At that time, there were only 5 GPS 
SV’s that the receiver had decided it would look for in the 
GPS constellation. However, the first four all had elevation 
angles greater than 60° while the fifth and non-simulated 
GPS SV was at 2.3°. Therefore the receiver used the four 
high elevation GPS SV’s and produced a PDOP solution 
that was above 1700. By changing the test simulation to 
simulate GPS SV’s that are visible down to 0°, the receiver 
used the fifth GPS SV and the anomaly never existed for 
that run. 

This particular test scenario was not a very proud mo- 
ment in the life of the project but it did uncover several 
important aspects of testing. First, this was the first test in 
which the receiver was tested for more than 15 hours. It 
was accidental that this test condition produced this prob- 
lem. Without the long test, there was a good chance this 
power cycle condition might never have been found. Sec- 





An example of the usefulness of several parameters plot- 
ted next to each other on the same page can be found in 
Figure 4 near the end of this paper. 

PACKETVIEW 

Test engineers at GSFC were able to capture several pa- 
rameters at the same time thru the use of a custom program, 
PacketView, written by Dr. Charles Campbell. Through the 
use of an initialization file, the user can specify commands 
that are to be sent either at the beginning of the test run, re- 
peatedly sent at a given rate, or sent once at a certain time 
into the run. Commands can be sent at any time during 
the run. Data from the receiver can be captured in binary 
or ASCII form. For testing of the SIGI receiver, all com- 
mands and output from the receiver were saved to a binary 
file. Certain parameters were routinely analyzed shortly af- 
ter the run (i.e. position, velocity, time, attitude, PDOP, 
C/No). These certain parameters were simultaneously cap- 
tured into an ASCII text file. The captured ASCII data was 
sorted into individual data files and then imported into post 
processing tools. 

The ability of PacketView data to be broadcast to other 
servers on our network was very beneficial during very long 
test runs. During a 12 and 24 hour test run, a quick look 
at the status of the receiver provided us a way to reduce 
the amount of wasted test time. This equates to a more 
immediate respond to a problem without having to actively 
monitor the receiver at all times during these long test runs. 
For example, during an overnight run, it was very easy to 
connect to a secure server at work from a remote location 
and monitor the status of the receiver. This greatly reduced 
the cost of having test engineers monitoring the receiver for 
three work shifts per day. PacketView also has a method of 
allowing only authorized users to send commands through 
the use of a password. 

TEST RESULTS 

The performance of the Force- 19 receiver is a success. The 
receiver provides the navigation and attitude performance 
required for the ISS. While only using three GPS antennas 
the receiver has demonstrated near requirement navigation 
and attitude performance. The most noticeable degrada- 
tion, while only using 3 antennas, was the loss of satel- 
lite coverage. The coverage during the three antenna runs 
tended to be 10 to 15% less than the coverage for the four 
antenna test runs. 

The receiver has been shown to provide the needed fea- 
tures required for the ISS. For example, the receiver can 
use a set of antennas which are not zenith pointing. The re- 
ceiver can accept GPS almanacs and aiding ephemeris for 
performing a warm start. During a warm start, the average 
time to a first position solution and then a first attitude solu- 
tion was 7:49 and 14:35, respectively. On average, during 
a cold start, the receiver produced it’s first position and at- 
titude solutions within 22:58 and 29:39, respectively. The 
warm start averages were based on 15 different almanac 


test cases using only 3 antennas while the cold start av- 
erages were based on 10 test runs of 5 different almanacs 
in a three antenna configuration. The cold start tests were 
considered to be much harder than the warm starts and 
thus did not require a full fifteen samples to met the de- 
sired confidence interval. It was successfully demonstrated 
that the user can decide which of the four available GPS 
antennas will be the master antenna for attitude determi- 
nation. Over the last 14 months, several hundred uploads 
of GPS almanacs were successful and over 20 different al- 
manacs were collected and downloaded from the receiver 
to be used in future warm start simulator test runs. It was 
also successfully demonstrated that the Force- 19 can be 
commanded to drop or attempt intentional track of a user 
defined GPS SV. The Force- 19 also successfully passed 
end-of-week rollover, August 21, 1999 rollover, and Y2K 
tests. It was also shown under a test environment that the 
Force- 19 would not attempt to track an unhealthy GPS SV 
based on almanac data. The receiver demonstrated that it 
could operate up to the 600 nautical mile altitude limit with 
three antennas (this is well above any possible ISS altitude). 
Parameters such as raw pseudorange, carrier phase mea- 
surements, GPS ephemeris, and almanac were shown to be 
made available to a user for future implementation of GPS 
relative navigation operations with the ISS. The receiver 
demonstrated that it could acquire and maintain position 
and attitude during and at a velocity of 12 ^ . However, 
the receiver could only maintain attitude at 5 ^ while the 
requirement was for 20 

A snapshot of the navigation and attitude performance 
from the Force- 19 receiver under ISS configurations during 
a GPS Signal Generator test run can be found in Figures 4 
and 5. 

In Figure 4 all error values are one sigma numbers in 
meters and the data are shown only when the current PDOP 
value is < than six. Also included in Figure 4 are the values 
of PDOP under six and then PDOP values for all instances 
of a position solution. Next the number of tracked GPS 
SV’s (according to the Force- 19 receiver) are plotted ver- 
sus time. This is followed by the receiver’s value of Carrier 
to Noise ratio (C/No). Finally, the signal strength (as pre- 
sented by the GPSSG) is plotted. It should be noted that the 
receiver can only measure C/No (a measure of how clean 
the signal is in the receiver). The coverage value listed at 
the top of the figure was computed while screening the po- 
sition solutions for PDOP < to six. Horizontal lines at 
PDOP value of six, at 35 dB Hz for C/No plot, and -95 
dBm and -65 dBm bound the range of acceptable values as 
listed in the ISS requirements document[3]. 

In Figure 5, the attitude determination performance (at- 
titude error) of the receiver is shown for each axis and a 
RSS of the attitude error is shown along the bottom plot in 
this figure. This plot also lists the attitude coverage. This 
attitude coverage is not screened for ADOP values of less 
than 0.06 (which is a screening criteria used in the ISS 
GN&C flight software). Along the top of each plot is the 
mean error value, standard deviation value. Root Mean 



Square of the error, 3 a value of the error (provided the 
data exhibits a Gaussian distribution), and the number of 
data points listed in the particular plot. 

The navigation performance, in the Position Error RSS 
plot of Figure 4, shows that the Force-19 is adequate with 
some caveats. For example, near the start of the test run, 
the Force- 19 has a large position error spike. This spike 
occurred even when the PDOP solution was < to six. This 
error spike occurred again about halfway through the test 
run. Notice that in the PDOP < six plot, that at that time 
a large amount of change has occurred with respect to the 
number of GPS SV’s which were tracked. This was not 
even a reflection on the amount of switching the GPS SV’s 
within the channels in the receiver and how that might af- 
fect navigation solution performance. Another note of in- 
terest should be the fact that the values of PDOP and num- 
ber of tracked PRN’s did not have a timetag. Several ap- 
proaches were taken in order to try and synchronize the 
PDOP and number of tracked PRN data with the position 
error data. During certain portions of the run, the GPS re- 
ceiver was too busy in order to service the telemetry and 
provide data to the user. This means that missing gaps of 
data can not be correlated to a timetag. This is a source of 
error that will have to be discussed with the customer and 
a course of action will then be decided upon. At this time, 
it is the conclusion of the test team that the lack of timetag 
may be one of the sources of the large position error spikes. 
Other possible sources of error are being investigated. 

Within Figure 4, the C/No and Truth Signal Strength 
plot demonstrates that the receiver can track signals at a 
reasonably weak signal level. Given the amplification of 
the preamplifiers used in the test setup, the GPS Signal 
Generator was tuned to provide the signal levels that were 
within the range of -95 dBm to -65dBm. These boundary 
values are represented as the solid horizontal lines in the 
Truth Signal Strength plot. The goal of the testing was to 
demonstrate that the receiver could track GPS signal at the 
C/No levelof atleast 35 dB-Hz. This is easily seen in the 
plot C/No where the data points reach and sometimes fall 
below the 35 dB-Hz threshold (shown as a solid horizontal 
line in the plot). 

The attitude performance plots, Figure 5, illustrate the 
attitude errors from the time the Force- 19 was powered on 
until the time the power was turned off. The Force- 19 
took 5 minutes and 56 seconds to calculate its first atti- 
tude solution. It should be noted that this was a four an- 
tenna test configuration with very little multipath injected 
into the simulator. The noticeable features of these plots 
were the greatest attitude error appeared along the Pitch 
axis and the attitude error spikes sometimes appeared on 
all three axes and sometimes only one axis. The fact that 
the worst axis for attitude performance was along the roll 
axis was not a coincidence. In fact, this axis lies along the 
short side of the retangular GPS antenna array. This short- 
est distance of the rectangle leads to a decrease in attitude 
accuracy. The second noticeable point from these plots was 


not so obviously explained. Attitude error spikes only oc- 
curring along one axis was typical behavior for Force- 19 
attitude performance. However, the ISS GN&C flight soft- 
ware will screen the attitude solutions using a long-term 
averaged value of an ADOP matrix to be < 0.06 ^-[5]. 
These types of errors are usually taken out at the expense 
of attitude coverage. It should also be noted that the errors 
do not contain a bias which makes the data suitable for use 
in the on-board ISS GN&C flight software attitude Kalman 
Filter. 
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